\chapter{Developing}

[student written chapter]

This chapter discusses the internal working of Vorpal that a developer will need to understand to modify or extend the language.  Because the implementation may change from version to version, only the high-level design is covered here.  For more details, consult the code documentation (built with Doxygen).

\section{Testing}
\label{Testing}
Vorpal includes a Python-based regression-testing environment. A test consists of two files: a \emph{.vorpal} source file and a \emph{.expected} output file. The \emph{.expected} file contains the Vorpal interpreter output expected from the corresponding test. A test will fail if the interpreter output does not match the contents of the \emph{.expected} file. All tests are stored in the \emph{test} directory.

\subsection{Running Tests}
Simply run the test script from your Vorpal directory:
\begin{lstlisting}
	$ ./test
\end{lstlisting}

\subsection{Creating a Test}
Write a Vorpal script that exercises the functionality you wish to test. For example, to test comments we could write the following script:
\vorpal
method() {
    #'This shouldn't be printed'.print() # Nor this
    'This should be printed'.print() # But not this
}
\end{lstlisting}

While the above script does not test every possible way in which a comment may be used, it serves our purpose. The above script should be given a cleat name such as \emph{tests/comments.vorpal}. We expect the script to produce the following output, when run by the Vorpal interpreter:
\begin{verbatim}
Vorpal
-----------
This should be printed
-----------
\end{verbatim} 

This output should be saved to the corresponding expected file, in this case \emph{tests/comments.expected}. The prefix of the filename (before the period) must match the prefix of the script file. You may use the stream operator to generate this \emph{.expected} file:

\begin{verbatim}
	$  bin/vorpal tests/comments.vorpal >> tests/comments.expected
\end{verbatim}

\noindent We may now run this test along with all the others, and an appropriate message will be displayed, indicating whether the test failed or passed:
\begin{verbatim}
	$ ./test
\end{verbatim}

% ===============================

\section{Lexer}
\emph{For implementation, please refer to} Lexer.cpp/h \emph{in the} src \emph{directory}\\

\noindent The lexer is responsible for scanning (see \emph{Tokenizer::Process}) and tokenizing (see \emph{Tokenizer::Tokenize}) Vorpal source code. 

At a high level, the lexer converts Vorpal source code into a series of tokens (each represented by a \emph{Token} object). Each token has a line number, a string representation, a length and a type. Types are defined in \emph{Compiler.h} and include: SIMPLE, OPERATOR, STRING, NUMBER, HAS, DOES, IF and WHILE.

The compiler will consume these tokens and invoke the assembler to create Vorpal machine code. This machine code will be executed by the Vorpal virtual machine.


% ===============================

\section{Compiler}
\emph{For implementation, please refer to} Compiler.cpp/h \emph{in the} src \emph{directory}\\

\noindent The compiler is responsible for parsing lexer output, and then assembling this output (using the assembler) into Vorpal machine code. Parsing is performed using a recursive descent parser, with grammar defined in \emph{Compiler.cpp}. In addition, the compiler is responsible for preprocessing (see \emph{Compiler::Preprocess}) the Vorpal source code to ensure code is enclosed in an anonymous method and insert the implicit self, where required. See Section \ref{Methods} for details on implicit self.

% ===============================

\section{Assembler}
\emph{For implementation, please refer to} Assembler.cpp/h \emph{in the} src \emph{directory}\\

\noindent The assembler translates compiler output to machine code. You may inspect the assembly code passed to the assembler (from the compiler) using the \verb@-d@ option on the Vorpal interpreter. For example:
\begin{verbatim}
    $ bin/vorpal -d HelloWorld.vorpal
    -----------
    Vorpal
    -----------
    code[13]:
        STR[12] = 'Hello World!'
        MESSAGE 1 2737915625
        DROP
        RETURN
        NOP
        NOP
        NOP
    -----------
    Hello World!
    -----------

\end{verbatim}



% ===============================

\section{VM}
\emph{For implementation, please refer to} VM.cpp/h \emph{in the} src \emph{directory}\\

\noindent The Vorpal virtual machine (VM) is responsible for executing compiled Vorpal machine code. Vorpal machine code is stored as distinct `cells'. Each cell may contain one of:
\begin{itemize}
	\item instruction[4]
	\item integer (32 bit)
	\item float
	\item hash value (32 bit)
\end{itemize}

\noindent The VM is implemented as a stack machine; it has no registers. Instructions and data are pushed and popped off a stack. 

\section{Garbage Collector}

The garbage collection technique used is reference counting and is implemented by the Handle class.  Never store or work with bare Object pointers; they should always be contained by a local \verb-Handle<Object>-.
